[HIP] [FlyDSL] Refactor gfx950 A16W16 GEMM with centralized policy selection and tuning - #5145
Merged
Merged
Conversation
xytpai
marked this pull request as draft
August 31, 2026 10:29
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
PR title tags & labels: |
xytpai
marked this pull request as ready for review
September 2, 2026 17:55
Collaborator
|
Approved and merged with following up requested. Don't forget to fix. |
jiacao-amd
pushed a commit
to jiacao-amd/aiter
that referenced
this pull request
Sep 12, 2026
Conflict was kimik3_bf16_tuned_gemm.csv only, and it was textual rather than semantic. ROCm#5145 retuned 184 rows of that file in place while this branch adds 7; the edits interleave in the same region, so git could not align the hunks. The two sets do not overlap. Every row ROCm#5145 retuned is outdtype=bfloat16 and every row this branch adds is outdtype=float32, and outdtype is part of the index get_GEMM_A16W16_config_ builds, so the two address disjoint keys. The merged-front GEMM writes an FP32 accumulator it slices router logits out of, which is why its rows are the fp32 ones. Resolved by key instead of by hunk: upstream's value for every row upstream has, this branch's 7 added rows kept, upstream's row order preserved. Verified that upstream neither reordered nor added/removed rows, that this branch only appends and never edits a pre-existing row, and that no added key collides with an upstream key -- if any of those had failed the resolution would not be mechanical. Result is 536 rows, 536 unique keys, no ragged rows. ROCm#5145 also refactored tuned_gemm.py, but only along the flydsl branch (get_flydsl_splitk_hgemm_kernel_params was renamed and the kernel parameter schema changed). This branch's rows are all libtype=hipblaslt and its code touches only hipb_create_extension, _hipb_mm and get_GEMM_A16W16_config, whose signature is unchanged. get_padded_m and the [None, 0, 1] lookup loop are also untouched, so the M=8 -> padded 16 assumption behind the M=16 row still holds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
splitk_hgemmandsmall_m_hgemmimplementations.Tuned Info
Across 2,186 updated model-config entries, the new configurations deliver a 1.055× geometric-mean speedup and a 3.06% median latency reduction.
FlyDSL HGEMM is selected for 1,554 of 2,186 entries (71.09%).
Reference